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Titel: Method for calibrating a camera^laser-unit in 
respect to a calibration-object 



Specification 

The present invention refers to a method for calibrating a 
camera-laser-unit in respect to at least one calibration- 
object disposed at a given position and orientation in a 
three-dimensional space. The camera-laser-unit comprises at 
least one laser (light-amplification by stimulated emission 
of radiation) and at least one camera- The laser and the 
camera are disposed in a given distance to one another. An 
optical axis of the laser and an optical axis of the camera 
include a given angle. The camera-laser-unit is adapted to 
record the location, form and/or dimensions of a measurement- 
object . 

Furthermore, the invention refers to a calibration-object 
used for calibrating a camera-laser-unit and disposed at a 
given position and orientation within a three-dimensional 
space. The camera-laser-unit comprises at least one laser and 
at least one camera. The laser and the camera are disposed in 
a given distance and an optical axis of the laser and an 
optical axis of the camera have a given angle. The camera- 
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laser-unit is adapted to record the location, form and/or 
dimensions of a measurement-object. 

Camera-laser-units of the kind mentioned above are well known 
in the state of the art. For example, such a camera-laser- 
unit is produced and sold by Perceptron, Inc., 47827 Halyard 
Drive, Plymouth, MI 48170, USA. Furthermore, such a camera- 
laser-unit and the use of such a unit is described in detail 
in the US-patents 4,645,348 and 6,166,811 both issued to 
Percept ron. Inc. A preferred embodiment of a camera-laser- 
unit of the kind mentioned above is described in detail in 
the German patent application 103 11 247, filed on March 14, 
2003 by the applicant of the present patent application. The 
publications of the two US-patents and of. the German patent 
application are all included by reference into the present 
specification. 

According to the methods known in the art, calibration of a 
camera-laser-unit is very complex and time-consuming. To 
begin with a first calibration-object has to be disposed in 
respect to the camera-laser-unit and the camera is calibrated 
in respect to the first calibration-object by, for example, 
making use of a Tsai algorithm well known in the state of the 
art. Thereafter, a second calibration-object has to be 
disposed in respect to the camera-laser-unit and the laser is 
calibrated in respect to the second calibration-object, too. 
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The second calibration^object in the state of the art 
typically comprises a planar object with an array of pins 
protruding from the plain. The pins represent a non-coplanar 
calibration-pattern used for calibrating the laser. The 
dimensions and distances of the pins are known. The problem 
is that the pins when illuminated by the laser or any other 
light source form shadows on the adjacent pins, the shadows 
rendering very complicated and even impossible an accurate 
calibration of the laser. 

Therefore, it is known to make the pins retractable and to 
extract them one by one from the plains of the cubic second 
calibration-object. After the extraction of each pin, a 
calibration routine is run and then the next pin is extracted 
with another calibration routine run thereafter, and so on. 
Thus, the calibration of the laser consists of numerous 
calibration routines. It may be well understood that in this 
way calibration of the laser is very cumbersome and requires 
a relatively long time. Typically, the calibration of the 
laser with the described method will take a couple of hours. 
Furthermore, the known calibration-method may be used only in 
laboratories or other places where the calibration assembly 
required for the calibration process may be build up. It is 
certainly not qualified for calibration of lasers during the 
intended use of the camera-laser-unit. 
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One way for simplifying the calibration of the laser could be 
to reduce the number of pins. However, reducing the number of 
pins would reduce accuracy of the calibration. 

It is an object of the present invention to simplify the 
process of calibration of a camera-laser-unit without 
reducing the available accuracy. 

This object is solved by a method for calibrating a camera- 
laser-unit comprising the features of claim 1. In particular, 
a method of the above mentioned kind is suggested comprising 
the steps of: 

selecting a calibration-object comprising at least 
two plains disposed in a given angle to each other 
and provided with a given non-coplanar calibration- 
pattern; 

disposing the calibration-object in respect to the 
camera-laser-unit in a given position and orientation 
in the three-dimensional space, wherein the 
orientation of the calibration-object is such that 
light emitted by the laser is visible on at least two 
plains of the calibration-object; 

calibrating the camera in respect to the calibration- 
object using a Tsai algorithm; 

activating the laser so it emits light visible on the 
at least two plains of the calibration-object; 
recording the light on the plains by the camera; 
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determining the laser^properties from the light 
recorded by the camera; and 

calibrating the laser according to the determined 
laser-properties . 

It is a main aspect of the present invention that the same 
calibration-object is used for the calibration of the camera 
as well as for the calibration of the laser. Furthermore, 
after calibration of the camera in respect to the 
calibration-object, the laser is calibrated in respect to the 
(already calibrated) camer.a. With other words, the camera is 
used for calibrating the laser. 

The whole procedure for calibrating a camera-laser-unit takes 
very little time and can be accomplished within few seconds. 
This makes it the ideal procedure for calibration of a 
camera-laser-unit during its intended field use. For example, 
the application of the camera-laser-unit (recording the 
location, form and/or dimensions of a measurement-object) 
could be interrupted for a short period, an industrial robot 
carrying the camera-laser-unit could move the unit to the 
calibration-object and dispose it in front of the object. 
Then the calibration process according to the present 
invention is run. Thereafter, the robot would move the 
camera-laser-unit away from the calibration-object back to 
the measurement-object and the intended use of the unit would 
be resumed. 
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According to a preferred embodiment of the invention it is 
suggested that calibrating the laser comprises the step of 
defining a relative position and orientation of the laser in 
respect to a coordinate frame associated with the 
calibration-object, wherein the coordinate frame associated 
with the calibration-object is in a given position and 
orientation in the three-dimensional space. 

Concerning the calibration of the laser, it is suggested that 
the light emitted by the laser is visible on the 
plains of the calibration-object as a line on each 
plain, the lines intersecting on a contact line of 
the two plains; and 

the laser-properties are determined from the lines 
recorded by the camera by means of a line detection 
- algorithm. 

Visible in the sense of the present invention means that the 
light emitted by the laser is visible to the camera. So this 
definition would comprise, for example, a case where the 
laser emits infrared (IR) light, invisible to the human eye 
but visible to an IR camera - 

In other words, three points can be defined for the 
calibration of the laser, the two end points of the lines on 
the two plains of the calibration-object and the point of 
intersection of the two lines. 
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Further, it is suggested that 

the light emitted by the laser is visible on the 
plains of the calibration-object as a line on each 
plain, the lines intersecting on a contact line of 
the two plains; 

a laser-plain is defined by the optical axis of the 
laser and the lines visible on the plains of the 
calibrat ion-ob j ect ; and 

in order to calibrate the laser according to the 
determined laser-properties, the position and 
orientation of the laser-plains in respect to a 
coordinate frame associated with the calibration- 
object is defined. 

Concerning the calibration of the camera it is suggested that 
the method comprises the step of defining a relative position 
and orientation of the camera in respect to a coordinate 
frame associated with the calibration-object, wherein the 
coordinate frame associated with the calibration-object is in 
a given position and orientation in the three-dimensional 
space. 

According to another embodiment of the present invention, it 
is suggested that a transformation matrix is defined 
depending on the relative position and orientation of the 
camera in respect to a coordinate frame associated with the 
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calibration-object, the relative position and orientation of 
the laser in respect to a coordinate frame associated with 
the calibration-object, and optionally on internal camera 
parameters. The picture taken from the measurement-object by 
the camera is superposed or multiplied with the 
transformation matrix in order to obtain the actual location, 
form and/or dimensions of the measurement-object. 

According to yet another embodiment of the present invention, 
it is suggested that the camera-laser-unit to be calibrated 
is grasped by an industrial robot and disposed in respect to 
the calibration-object in a given position and orientation in 
the three-dimensional space, wherein the orientation of the 
camera-laser-unit is such that light emitted by the laser is 
visible on at least two plains of the calibration-object. 

Furthermore, the object of the present invention is solved by 
a calibration-object of the above-mentioned kind 
characterized in that it comprises two plains disposed in a 
given angle to each other, provided with a non-coplanar 
calibration-pattern on each plain comprising an array of 
features, and the calibration-object used for calibration of 
the camera as well as for calibration of the laser. The array 
of features represents a grid of calibration points used for 
calibrating the camera as well as the laser of the camera- 
laser-unit . 
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According to a preferred embodiment of the present invention 
it is suggested that the angle between the two plains is a 
right angle. The light emitted by the laser is visible on the 
plains of the calibration-object as a line on each plain, the 
lines intersecting on a contact line of the two plains. In 
this way the maximum possible volume in space can be 
calibrated. 

The features of the calibration-object may comprise recesses, 
in particular cavities with a circular cross section, or 
prints on the plains of the calibration-object. The features 
may have any shape or design. For example, the features may 
comprise rectangular recesses or prints with a square or a 
circular cross section viewed from above. The prints on the 
plains have a different color or a different surface than the 
plains of the calibration-object, so the features and the 
calibration pattern can be detected by the camera during the 
calibration process. 

Further embodiments as well as further advantages of the 
present invention are outlined in the following description 
of the enclosed figures . 

Fig. 1 shows a schematic view of a preferred embodiment of 
a calibration setup for calibrating a camera-laser- 
unit according to the present invention; 
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Fig. 2 shows a schematic view of a preferred embodiment of 
an assembly for recording the location, form and/or 
dimensions of a measurement-object by means of the 
calibrated camera-laser-unit ; 

Fig. 3 shows a calibration setup for the camera-laser-unit 
in detail; 

Fig- 4 shows a flow chart of a preferred embodiment of a 
method for calibrating a camera-laser-unit 
according to the present invention; and 

Fig. 5 shows an example for a calibration-image. 

The present invention refers to a camera-laser-unit for 
recording the location, form and/or dimensions of a 
measurement-object. The camera-laser-unit shown in figure 2 
as a whole is designated with reference sign 1. It comprises 
mounting means 2 with a camera 3 and a light source 4, in 
particular a laser, mounted thereon. The laser 4 illuminates 
part of the measurement-object 5 and the camera 3 takes an 
image of the illuminated measurement-object 5. The camera 3 
comprises a CCD (charged coupled device) -chip 6 or other 
means (e.g. a CMOS-chip) for converting optical signals of 
the image into electrical signals for further processing 
within the camera-laser-unit 1. Furthermore, the camera 3 
comprises an optical system 7, in particular a lens, for 
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reproducing the image on the CCD^chip 6. The camera 3 and the 
laser 4 are disposed on the mounting means 2 such that an 
optical axis 8 of the camera 3 intersects an optical axis 9 
of the laser 4 at a measurement point 10. The measurement 
point 10 is not necessarily disposed on a contact line 
between the two plains 14, 15. Preferably the optical axis 8 
and 9 are disposed in an angle a of 45*" in respect to one 
another. The laser 4 generates a line 11 on the measurement 
object 5, which is recorded by the camera 3 and reproduced on 
the CCD-chip 6 by the lens 7. 

For example, a camera-laser-unit of the mentioned kind is 
produced and sold by Perceptron, Inc., 47827 Halyard Drive, 
Plymouth, MI 48170, USA. Furthermore, such a camera-laser- 
unit and the use of such a unit is described in detail in the 
US-patents 4,645,348 and 6,166,811 both issued to Perceptron, 
Inc. A preferred embodiment of a camera-laser-unit of the 
kind mentioned above is described in detail in the German 
patent application 103 11 247, filed on March 14, 2003 by the 
applicant of the present patent application. The publications 
of the two US-patents and of the German patent application 
are all included by reference into the present specification, 
in particular for more detailed information on possible 
structures of camera-laser-units and on the process of 
determining the location, form and/or dimensions of a 
measurement-object by means of a camera-laser-unit. 
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However, before the camera-laser-unit 1 can be used for 
recording the location, form and/or dimensions of a 
measurement-object 5, it has to be calibrated. During the 
calibration process the camera 3 and the laser 4 are put into 
relationship with the coordinates of a calibration-object 12 
(see figure 1) and a three-dimensional space 13/ if the 
calibration-object 12 is disposed at a given position and 
orientation within the three-dimensional space 13- In 
particular, camera parameters are determined and a relative 
position of one or more laser plains is defined with respect 
to a coordinate frame 19 related to the calibration-object 
12. A laser plain is defined by the optical axis 9 and the 
line 11 of light emitted by the laser 4 (see figure 2 or 3) . 
Calibration of the camera-laser-unit 1 is necessary in order 
to allow the camera-laser-unit 1 to determine absolute values 
for the location, form and/or dimensions of the measurement- 
object 5. 

Figure 3 shows the calibration-object 12 according to the 
present invention with its two plains 14, 15 facing the 
camera-laser-unit 1. Any object 12 comprising two calibration 
plains 14, 15 is suitable for the calibration process 
according to the invention. A non-coplanar calibration- 
pattern comprising an array of recesses 17 is disposed on the 
two plains 14, 15. Of course, the calibration-object 12 can 
comprise an array of any kind of features, such as simple 
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prints on the plains 14, 15 of any desired shape and cross 
section, instead of the recesses 17. 

In the embodiment shown in figure 3 the calibration-pattern 
comprises a six by six array of 36 grid-like disposed 
recesses 17 on each plain 14, 15. Each recess 17 of the 
calibration-pattern is designed as a cavity having a circular 
cross section viewed from above. The same calibration-object 
12 is used for calibration of the camera 3 as well as for 
calibration of the laser 4. The two plains 14, 15 are 
disposed in a known angle p, preferably 90°. The closer to 
perpendicular the two plains 14, 15 are, the larger is the 
volume over which the calibration is valid, so larger angles 
P could be used if volume is not a critical factor. 

The present invention refers to a particularly advantageous 
process for calibrating the camera-laser-unit 1 in respect to 
the calibration-object 12 disposed at a given position and 
orientation within the three-dimensional space 13. The 
process comprises the following steps: 

Selecting a calibration-object 12 comprising at least 
two plains 14, 15 disposed in a given angle p to each 
other and provided with a given non-coplanar 
calibration-pattern (see figure 3) . Preferably, the 
angle p is 90°. 
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Disposing the calibration-object 12 in respect to the 
camera-laser-unit 1 in a given position and 
orientation in the three-dimensional space 13, 
wherein the orientation of the calibration-object 12 
is such that light emitted by the laser 4 is visible 
for the camera 3 on at least two plains 14, 15 of the 
calibration-object 12 . 

In a first step, calibrating the camera 3 in respect 
to the calibration-object 12 using a Tsai algorithm. 
The Tsai algorithm is well known for camera 
calibration. It is described in detail in M. Tapper, 
Ph. J. McKerrow, J. Abrantes: "'Problems Encountered 
in the Implementation of Tsai's Algorithm for Camera 
Calibration", Proc. 2002 Australasian Conference on 
Robotics and Automation, Auckland, 27-29 November 
2002. Concerning the camera calibration by means of a 
Tsai algorithm reference is made to this document. 
In a second step calibrating the laser 4 in respect 
to the (already calibrated) camera 3, and activating 
the laser 4 in order to make it emit light (line 11) 
visible as two lines 11a, lib on the two plains 14, 
15 of the calibration-object 12, 

recording the light (lines 11a, lib) on the plains 
14, 15 by the camera 3 and reproducing the image 
(lines 16a, 16b) on the CCD-chip 6 by the camera lens 
7, 
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determining the laser-properties from the light 16 
recorded by the camera 3, and 

calibrating the laser 4 according to the determined 
. laser-properties. 

The laser-properties are determined from the properties 
(length and angle y) of the lines 16a^ 16b reproduced on the 
CCD-chip 6 of the already calibrated camera 3 by using a line 
detection algorithm. The two lines 11a, lib can be easily 
reconstructed in space from the images 16a, 16b using the 
camera parameters and the known geometry of the calibration 
object 12. Once the two three dimensional lines 11a, lib are 
determined, the information is adequate to reconstruct the 
laser plain (s) . 

According to the present invention, calibration of the 
camera-laser-unit 1 is performed in two steps: 

First, standard camera calibration is performed to 
define the internal parameters of the camera 3 and 
the position of the camera 3 with reference to a 
coordinate system. 

Second, the position of the laser plain (defined by 
the optical axis 9 and the line 11) is defined with 
reference to the same coordinate system. 

These two sets of parameters - camera parameters and laser 
plain equations - provide all the information necessary for 
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performing projection and reconstruction using the camera- 
laser-unit 1 (e.g. transformation from image pixels 18 of the 
CCD-chip 5 (see figure 3) into millimeters (mm) and vice 
versa . 

Making reference to the flow chart of figure 4, the method 
for calibrating the camera-laser-unit 1 according to the 
present invention will now be described in detail. The first 
step of the calibration process is to calibrate the camera 3 
(step 20) using an image 21 and the geometry 22 of the 
calibration object 12 and nominal camera parameters 23. The 
calibration of the camera is done by using a non-coplanar 
Tsai calibration algorithm. As a result of the camera 
calibration the camera parameters 24 are obtained. 

The second step of the calibration process is to determine 
the laser plain (s) (step 25) defined by the optical axis 9 
and the line 11 of light emitted by the laser 4. This can be 
accomplished by detecting the points belonging to the laser 
lines 11a, lib visible on the calibration-object 12. The 
laser line(s) 11a, lib are extracted (step 26) from the laser 
line image 27 taken by the camera 3. The geometry 22 of the 
calibration object 12, the camera parameters 24 and the 
extracted laser lines 26 are used for determining the laser 
plain (s) 25. As a result of the determination of the laser 
plains (s) the laser plain (s) 28 are obtained. Output 29 is 
calculated from the camera parameters 24 and the laser 
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plain (s) 28. The output is, for example, a transformation 
matrix. 

Figure 5 shows an example for a calibration-image comprising 
two laser plains each with lines 11a and lib on the plains 14 
and 15 of the calibration object 12 respectively. Provided 
that the scene illumination is not too bright (to avoid 
saturation) and that the surface of the calibration object 12 
is not too reflective, the laser lines 11a and lib can be 
easily distinguished. 

The detection of the laser lines 11a and lib comprises the 
following steps: 

Automatically defining or detecting the regions in 

the image recorded by the camera 3 covered by each 

plain 14, 15; 

For each region: 

o scanning the region horizontally or vertically, 

depending on the relative positions of camera 3 and 
laser 4 and detecting the brightest pixel along 
each line or column. For multiple laser plains (see 
figure 5), multiple points can be detected, 
provided that some estimation for the minimum and 
maximum distance between the lines is known (in 
pixels) . This estimation can either be defined by 
the user, or calculated automatically from the 
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angle between two adjacent laser plains and the 

calibration plains' 14, 15 positions, 
o For each detected pixel, detecting the center of 

intensity on its neighbourhood, using the weighted 

average of pixel intensities, 
o For each laser plain, fitting a line on the 

detected points, using a linear regression 

algorithm. 

o Calculating the distance of each detected point 
from all detected lines 11a, lib. If the point is 
closer to some other line 11a, lib, then this point 
is misclassif ied, so it has to be moved to the 
correct set of points. 

One approach to define a laser plain once a pair of laser 
lines 11a, lib has been extracted (step 26), is to 
reconstruct the lines in three dimensional space, using the 
camera calibration parameters 24 and the plain equations of 
the calibration plains 14, 15. The two reconstructed lines 
have adequate information to determine the plain of the laser 
4 in the same coordinate system. A second approach that has 
proved to be more accurate, is to reconstruct all detected 
points (using the camera parameters 24 and the equations of 
the calibration plains 14, 15), and iteratively fitting a 
plain to the reconstructed set of points, discarding at each 
step the most distant points. The position and orientation of 
the two plains 14, 15 can be easily defined with respect to a 
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conunon coordinate system^ e.g. centered at the point of 
intersection 10 of the two lines lla, lib. 

As described above, calibration of a camera-laser-unit 1 with 
any number of laser plains can be performed using a three 
dimensional calibration-object 12, comprising at least two 
calibration plains 14, 15. Of course, the complexity of the 
procedure would increase if objects 12 with more than two 
plains were considered- The main advantages of the method 
according to the present invention are: 

The same camera model that is used for the 

calibration of the camera 3 can be used for the 

calibration of the laser 4 allowing a straight 

forward integration of the laser calibration into 

existing camera calibration processes - 

Calibration of the camera 3 is performed by using an 

already existing, well known and highly tested Tsai- 

based algorithm and appropriate software. 

The topology of the calibration-object 12 is easy to 

construct with very high accuracy. 

High accuracy of calibration can be guaranteed over 
an entire volume covered by the calibration object 
12. 

The camera-laser-unit 1 can be easily and directly 
calibrated in any coordinate frame 19 (if the 
position of point of origin 0 of the frame 19 is 
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known in the global coordinate frame 13) , without the 
need for time-consuming alignment processing. 



